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Abstract 

A conceptual framework involving partition functions of normal factor graphs is introduced, 
paralleling a similar recent development by Al-Bashabsheh and Mao. The partition functions of 
dual normal factor graphs are shown to be a Fourier transform pair, whether or not the graphs 
have cycles. The original normal graph duality theorem follows as a corollary. 
' Within this framework, MacWilliams identities are found for various local and global weight 

generating functions of general group or linear codes on graphs; this generalizes and provides 
£\| ' a concise proof of the MacWilliams identity for linear time-invariant convolutional codes that 

was recently found by Gluesing-Luerssen and Schneider. Further MacWilliams identities are 
developed for terminated convolutional codes, particularly for tail-biting codes, similar to those 
studied recently by Bocharova, Hug, Johannesson and Kudryashov. 
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1 Introduction 

00 . 

Linear codes defined by graphical models have become the central subject of modern coding theory. 
Moreover, the topic of "codes on graphs" has proved to have rich connections to such cognate fields 
as system theory and statistical physics. 

Duality has long been a prominent theme in the study of linear codes. Duality theory often 
Q^ [ yields simple, powerful, and surprisingly general results. Some problems become much simpler in 

the dual domain. 

A fundamental duality result in the field of "codes on graphs" is the normal graph duality 
theorem of [5]. By imposing certain natural "normal" degree restrictions on graphical models, 
which are in fact not at all restrictive, the normal graph duality theorem shows how a graphical 
model for a dual linear code C 1 - may be obtained by local dualization of a graphical model for a 
linear code C, whether or not the graph has cycles. 

At the same time as [5] (in an adjacent paper in the same special issue), the conceptual frame- 
work of "factor graphs" was introduced by Kschischang, Frey, and Loeliger [11] to unify various 
styles of graphical models such as Tanner graphs, Bayesian networks, Markov random fields, Kalman 
filtering, and so forth, and the various computational algorithms that have been developed inde- 
pendently in these various fields. Subsequently, the "normal" restriction has been adopted to some 
extent in the factor graph literature, particularly by Loeliger and his co-authors (see, e.g., [T2 , 13J). 

Mao and Kschischang [15J have developed a general duality theory for factor graphs, particularly 
by introducing "convolutional" factor graphs as duals to conventional multiplicative factor graphs. 
By specializing to normal graphs, they proved the normal graph duality theorem as a corollary. 
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Very recently, Al-Bashabsheh and Mao pQ have shown that the normal graph duality theorem 
and Valiant's "holographic" algorithms can both be united within a common framework, which they 
call "normal factor graphs and holographic transformations." We have collaborated intensively with 
the authors of that paper and with the Associate Editor of both (P. Vontobel) while writing this 
paper, and have tried to achieve as much commonality as possible in our presentations. 

As in p] , we regard normal factor graphs as representations of realizations that involve external 
variables, internal variables and factors. All variables are vector spaces over a given finite field 
F, and all factors are complex- valued. Each factor involves some subset of the variables, with the 
following "normal" degree restrictions: each external variable is involved in precisely one factor, and 
each internal variable is involved in precisely two factors. As shown in [5], any realization may be 
straightforwardly "normalized" by a simple replication procedure that does not affect the topology 
of the associated graphical model. In the corresponding normal factor graph (NFG), factors are 
then represented by vertices, internal variables by ordinary edges of degree 2, and external variables 
by edges of degree 1 ("half-edges," "dangling edges," "dongles"). 

We define the partition function of such a normal factor graph as the sum over all internal 
variables of the product of all factors. (In [I], this is called the "exterior function" of the NFG.) 
In particular, if all factors are indicator functions of local linear constraint codes over F, then the 
partition function is (up to scale) the indicator function of a global linear code C. 

With pQ , we believe that the conceptual framework of representing sums of products as partition 
functions of normal factor graphs is an important paradigm that, surprisingly, does not seem to 
have been discussed very explicitly in the previous factor graph literature. We therefore present 
this framework in some generality in Sections 2 and 3. 

In Section 3, we prove a general normal factor graph duality theorem (also proved in [1]), which 
shows that the partition functions of a normal factor graph and its dual are a Fourier transform 
pair, up to scale. Specializing to indicator functions of linear codes, we obtain as a corollary a 
result that is equivalent to the normal graph duality theorem of pjj. 

In Sections 4 and 5, we present further applications that were stimulated by several recent 
results on Mac Williams identities for linear convolutional codes [9] [TO] [2] . 

For linear block codes, Mac Williams identities are classical duality results that relate weight 
generating functions of linear codes and their duals [14j . It was shown more than thirty years ago 
by Shearer and McEliece [16] that there is no Mac Williams identity for the usual weight generating 
function (the free distance spectrum) of a convolutional code. 

However, Gluesing-Luerssen and Schneider (GLS) have recently formulated [9] and proved [TO] 
a Mac Williams identity that involves the Hamming weight adjacency matrix (HWAM) of a linear 
time-invariant convolutional code over a finite field and the HWAM of its dual code. 

In Section 2, we show how weight generating functions of various types may be naturally 
represented as partition functions of normal factor graphs. Furthermore, for a convolutional code, 
it is natural to replace a weight generating function (WGF) by a weight adjacency matrix (WAM). 

In Section 4, we then apply our normal factor graph duality results to prove an appropriate 
MacWilliams identity between a WGF (or WAM) of a local linear code and a WGF (or WAM) 
of its dual. We consider exact, complete, and Hamming weight generating functions. This gives a 
concise proof of the GLS result, and generalizes it to arbitrary group codes defined on graphs; e.g., 
linear block codes defined on trellises, linear tail-biting codes, or trellis codes over abelian groups. 
A preliminary version of these results was presented in [6] . 
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Bocharova, Hug, Johannesson, and Kudryashov [2] have recently proved a Mac Williams identity 
that holds for truncations of a convolutional code C and its orthogonal code C^~. As the truncation 
length becomes large, they obtain approximations to the free distance spectra of C and C^. 

In Section 5, we develop Mac Williams identities for distance distributions of various kinds of 
terminated convolutional codes, particularly tail-biting terminated codes. A preliminary version of 
these results was presented in [7]. These results effectively answer the original question posed by 
Shearer and McEliece |16| . 

2 Codes, Realizations and Graphical Models 

In this section we review linear codes, realizations of codes, and their graphical models. We start 
with the development and notation of [5], but then transition to normal factor graphs rather than 
normal graphs. We develop the general framework of partition functions of normal factor graphs. 
Finally, we show how weight generating functions are naturally represented in this framework. 

2.1 Linear codes, realizations and normal graphs 

In this paper we will be concerned with linear codes over a finite field F. Everything generalizes to 
group codes over finite abelian groups, but for simplicity we will restrict attention to linear codes. 

A linear code C over F is defined over a discrete index set X4 and a set {Ak : k £ X4} of symbol 
alphabets Ak, each a finite-dimensional vector space over F, and thus finite. The code C is then a 
subspace of the Cartesian-product vector space A = HkeiAk, called the symbol sequence space. 

In general, the index set Z4 may be infinite; however, in this paper we will assume for simplicity 
that Z4 is finite, so that the symbol sequence space A is finite. For convolutional codes, this 
assumption may be justified by considering an infinitely long convolutional code as a limit of a 
sequence of finitely long terminated codes; see Section 5. 

A realization of a code C is a concrete implementation or characterization of it. For instance, a 
parity-check realization of a linear (n, k) code C characterizes it as the set of all a € F n that satisfy 
a set of n — k parity-check equations. A Tanner graph is a graphical model of such a realization. 

More generally, a behavioral realization of C involves not only the set {Ak : k £ Z4} of symbol 
alphabets, but also a set {Sj : j £ X$} of auxiliary alphabets, often called state spaces, indexed by 
a state index set X$, and a set {Cj : i £ Xq) of local constraint codes C{ indexed by a constraint 
index set Xq , where each constraint code Cj involves some subsets aj and Sj of the symbol and state 
variables, respectively. In a linear behavioral realization, each state space Sj and each constraint 
code Ci is a vector space over F. We define the state sequence space as S = Hj<=z s Sj. 

The full behavior of the realization is the set 33 of all pairs (a, s) £ A x S such that all constraints 
are satisfied; i.e., (aj,s,) £ Cj,Vi £ Xq. The code C generated by the realization is then the set of 
all symbol sequences a £ A that appear in some (a, s) £ 55. 

For example, in a conventional state realization of a linear code C, the symbol index set I4 is a 
conventional discrete time axis, namely the set of integers Z, or a sub interval of Z. The state index 
set X$ may be thought of as the set of times that occur between consecutive pairs of times in Z4, 
and the state time preceding symbol time k £ X4 is conventionally also denoted by k £ X$. The 
constraint codes {Ck ■ k £ 1^} are linear codes indexed by the symbol index set X4, and specify the 
set of all valid (sk, ak, Sfc+i) transitions; i.e., for each k £ Z4, Ck is a subspace of the vector space 
Sk x Ak x Sk+i- The full behavior *B of the realization is the set of all symbol/state trajectories 
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(a, s) such that (sk, a&, s/c+i) is a valid transition in Ck for all k G I4. The code C generated by the 
realization is the set of all symbol trajectories a that appear in some (a, s) £ 33. 

A normal behavioral realization is defined as a behavioral realization in which every symbol al- 
phabet is involved in precisely one constraint code, and every state space is involved in precisely two 
constraint codes. Thus a conventional state realization is normal. As shown in [5], any behavioral 
realization may be straightforwardly converted to a normal realization by the "normalization" pro- 
cedure that will be described in the next subsection, without essentially increasing the complexity 
of the realization. 

A normal behavioral realization has a natural graphical model, called a normal graph, in which 
each constraint code Ci corresponds to a vertex, each state space Sj (which by definition is involved 
in two constraints) corresponds to an edge connecting the two corresponding constraint vertices, 
and each symbol alphabet Ak (which by definition is involved in one constraint) corresponds to 
a leaf or "half-edge" connected to the corresponding constraint vertex. For example, Figure Q] 
shows the normal graph corresponding to a conventional state realization, which is a simple chain 
graph. Here vertices are represented by square boxes, and the "half-edges" corresponding to symbol 
alphabets are represented by special "dongle" symbols. 









Ak+i 




Ak+2 




Sk 


T 


S k +\ 


T 


Sk+2 


T 



















Figure 1: Normal graph of a conventional state realization. 
2.2 Normal realizations and normal factor graphs 

In this paper, we will mostly represent codes (and weight generating functions of codes) by normal 
factor graphs and their partition functions. A partition function will be defined by 

• a set A = {Ak, k € I j\} of external variables Ak with alphabets Ak] 

• a set S = {Sj,j £ Is} of internal variables Sj with alphabets Sj] 

• a set f = {fi,i € Ip} of complex-valued factors fi, each factor /, involving subsets Aj C A 
and Si C S of the sets of internal and external variables. 

All sets are assumed to be finite, and all variable alphabets are assumed to be finite-dimensional 
vector spaces over some finite field F. We call the Cartesian-product alphabet A = Yl k Ak the 
external variable configuration space, and S = Y\j Sj the internal variable configuration space. 

The partition functior0 (or "external function" pQ) defined by these elements is the function 
Z : A — > C of the external variables that is given by the following sum of products: 

Z(a) = II /i( a i> s *)' a G -4; 

i.e., the sum over all internal variable configurations of the product of all factors. 

1 In physics, a partition function is usually defined as a sum over internal configurations, and there are no external 
variables, so this usage of "partition function" extends the usual terminology of physics. 
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A partition function Z(a) may in general be given by many different sum-of-products forms, 
which we will call realizations. We will say that two realizations of the same partition function are 
equivalent. 

A realization will be called normal if each external variable is involved in precisely one factor, 
and each internal variable is involved in precisely two factors. As noted in [5], any realization may be 
converted to an equivalent normal realization by the following simple normalization procedure: 

• For every external variable Ai, if A, is involved in p factors, then define p replica variables 
An, 1 < £ < p, replace Ai by An in the £th factor in which Ai is involved, and introduce one 
new factor, namely an equality indicator function <I> = (aj, {an, 1 < t < p}); i.e., the (0,1)- 
valued function that equals 1 when a, = an = • • • = a^, and that equals otherwise. Thus 
each replica variable An becomes an internal variable that is involved in precisely two factors, 
while Ai remains an external variable that is involved in only one factor, namely the equality 
indicator function. (If p = 1, then this conversion need not be performed.) 

• For every internal variable Sj, if Sj is involved in q > 2 factors, then define q replica variables 
Sjt, 1 < ^ < <?> replace Sj by Sjt in the £th factor in which Sj is involved, and introduce one 
new factor, namely an equality indicator function &=({sj£,l < I < q}). Thus each replica 
variable Sjt becomes an internal variable that is involved in precisely two factors. (If q = 2, 
then this conversion need not be performed. If q = 1, then multiply the partition function by 
a dummy factor l(s&) which is equal to 1 regardless of the value of Sk-) 

Evidently this normalization procedure does not change the partition function Z(a). Also, as 
can be seen from [5], the normal factor graph that represents the normal realization is essentially 
unchanged from the bipartite factor graph that represents the original realization. 

A normal realization is represented by a normal factor graph (NFG)U As in a normal graph, 
ordinary edges represent internal variables and half-edges represent external variables, but now 
vertices represent factors rather than constraints. The partition function of the NFG is the partition 
function Z(a) of the associated realization. 

In this paper we will mostly adopt the convention of ignoring multiplicative scale factors a > 
in partition functions. We will say that the partition function is equal to Z{a), up to scale; that is, 
the relative weights Z(a) for the various configurations a E A are correct, but the absolute value 
may not be. In many applications the absolute scale factor is not important; but if it is, then it 
can be reconstructed from the constituent factors /j. 

A normal graph representing a code C may be converted to a normal factor graph representing 
the indicator function <3?e of C as follows. Let each local constraint code Ci be replaced by the 
(0, l)-valued indicator function <j?c,: of Cj. Then, for a given external variable assignment a E A, 
the partition function of the graph — i.e., the sum over all s E S of the product of all local 
constraint code indicator functions — is the number of internal variable assignments s E S such 
that (a, s) satisfies all local constraints. For a linear code C, it is often true that a unique s E S is 
determined by each a E C; e.g., when the graph is cycle- free and the realization is minimal. But 
in any case, by linearity, the same number of state sequences correspond to every a E C, namely 

2 Loeliger et al. [121113] define "Forney-style factor graphs" just as we have defined normal factor graphs. However, 
in the usual factor graph framework, such a graph represents simply the product of the factors, rather than a sum 
of products. Loeliger et al. do also consider sums of products within "boxes," namely graph fragments enclosed by 
dashed lines. 
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|{s G S : (0, s) G 23}|. Therefore, up to scale, the partition function of the NFG is equal to the 
(0, l)-valued indicator function i.e., 

For example, Figure [2] shows the normal factor graph corresponding to the normal graph of 
Figure [TJ 
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Figure 2: Normal factor graph of a conventional state realization. 



2.3 Weight generating functions 



Weight generating functions are standard tools of combinatorics. We will later consider various 
kinds of weight generating functions, but for linear codes over finite fields, the simplest and com- 
monest kind of weight generating functions are Hamming weight generating functions. 

The Hamming weight generating function (HWGF) of a linear code C defined over a finite index 
set X4 with symbol sequence space A = Y\k e x A ^-k is the polynomial 



En- 



where wniak) 1S the Hamming weight of the symbol aj. 6 4. Thus the coefficient of x w in g^{x) 
is the number of codewords a G C that have Hamming weight w. 

This sum-of-products expression for g^{x) suggests that a HWGF might be represented as the 
partition function of a normal factor graph. (This idea was briefly mentioned in |1H Example 13].) 
Indeed, since 



g?(x) = £ * c (a) 



aeA 



it follows that g^{x) is the partition function of the simple normal factor graph of Figure Eta), in 
which the two functions $c( a ) an d connected by the internal variable A. 
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r w H (ai) 
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Figure 3: Normal factor graphs of the HWGF g^{x) of a code C C A. 

Or, since w^(a) = X^fcex^ w H(o>k) is an additive function, g^{x) may alternatively be repre- 
sented as the partition function of the normal factor graph of Figure EJb) , in which the function 
3>c( a ) is connected by an internal variable A^ to the function x WH ( ak ' for each k G X4. 
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Now, more generally, if the indicator function $c of a linear code C is the partition function of 
any normal factor graph, then a normal factor graph for g^ (x) may be obtained by replacing the 
symbol half-edge associated with A k in that graph by an ordinary edge connected to the function 
x w H (a k ) Q f A k f or eac h k £ X4. (Or, if wnip-k) = Yli w H{a'ki) for some components a k g of a k , then 
A k and x WH ( afc ) may be broken down into their components A k i and x WH ^ au ' as in Figure [3](b).) 

For example, for a normal factor graph of a conventional state realization as in Figure EJ the 
HWGF Qq(x) is the partition function of the normal factor graph shown in Figure [H 
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Figure 4: Normal factor graph of the HWGF <?<?(x) of the code C of Figure 2. 

Such a representation of g^(x) may be further simplified by summing over the variables A k 
for each k S X4, which have now become internal variables. For a code represented by a conven- 
tional state realization as in Figure HI this results in a normal factor graph as in Figure [5j Here 

i s k,Sk+i) is what is called [HI [TO] the Hamming weight adjacency matrix^ (HWAM) of C k : 
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Figure 5: Normal factor graph resulting from summing over each A k in Figure 4. 
In equations, the equivalence of the partition functions of Figures [4] and [5] follows from 

9c(x) « EE II *e k (s k ,a k ,s k+1 )x w »^ 

seS aeA k€X A 

= E II E *C k (s k ,a k ,s k+1 )x w »^ 
= E II A%( Sjb s fe+1 )0r). 

The last expression may be recognized as simply the product • • • Ac k (x)Ac k+1 (x)Ac k+2 (x) • • • of the 
HWAMs, using the usual rules of matrix arithmetic. 

Example 1 (binary linear block code). Consider the (8,4) binary linear first-order Reed-Muller 
code C, which has the conventional four-section state realization (trellis) shown in Figured] |5j. 

3 Perhaps this should have been called a "Hamming weight generating function adjacency matrix," but we prefer 
to follow the established terminology. 
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Figure 6: Four-section state realization of (8, 4) binary first-order Reed-Muller code. 
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Let us now consider the Hamming weight generating function of a fragment of a normal factor 
graph realizing a code C, consisting of a subset of the constraint codes Cj. An external (symbol) 
variable A\~ may or may not be involved in one of these constraint codes; if it is, we attach an 
enumerator function 

x WH M as above. An internal (state) variable Sj may appear in two, one, or 
none of these constraint codes: if it appears twice, then we consider it to be an internal variable of 
the fragment and sum over itS if it appears once, then we consider it to be an external variable of 
the fragment. In other words, the HWGF of the fragment is the partition function 

gf(x)(s ext )^Yl II E'M^Si^H 

Sint i&Xj^J Si 

where s ext and Si n t are the external and internal state variables with respect to the fragment, and 
Z_4 t C Z4 is the subset of indices of constraint codes C, that appear in the fragment. 

Example 2 (N sections of a linear convolutional code). Consider a linear time- invariant convolu- 
tional code C, in which all symbol alphabets Ak, state spaces Sk, and constraint codes Ck do not 
actually depend on the time index k £ Z. Then the HWAM A^ (x) does not depend upon k, so we 
write it simply as A(x). Consider a fragment of a graph for C consisting of N consecutive trellis 
sections, over a time interval [k, k + N). Then, relative to this fragment, the external variables are 
Sk and S'fc+Ar, and the HWAM of the fragment is simply the iVth power A N (x) of the HWAM A(x) 
of a single section. □ 



3 Duality 

The central duality result of |5j is the normal graph duality theorem: given a normal realization of 
a code C, the dual normal realization generates the dual (orthogonal) code C . We will now give a 
concise proof of this result, as a corollary of a general duality theorem for normal factor graphs. 

4 Sometimes physicists use the Einstein summation convention: in a tensor product, variables that occur twice 
are implicitly to be summed over. This convention would evidently be useful for partition functions of normal factor 
graphs. 
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3.1 Fourier transforms 



We first recall the theory of Fourier transforms over finite fields, generally following [4J. 

Let the prime p be the characteristic of the finite field F; i.e., p is the least positive integer such 
that pa = for all a 6 F. 

If A is a vector space over F, then its dual space A may be defined as the set of all homomor- 
phisms a : A — > Z p . It follows from Pontryagin duality theory [4] that A is a vector space over 
F of the same dimension as A, and that the dual space to A is A, with a(a) defined as d(a). We 
may then define the inner product (a, a) as a(d) = a(a) for all a 6 A, a 6 A. The inner product so 
defined has the usual properties; e.g., (a, 0) = (0, a) = 0, (a, a + a') = (a, a) + (a, a'), and so forth. 

For example, if A is the field F, then the additive group of F is isomorphic to (Z p ) m for some 
integer m. The dual space A may also be taken as F ~ (Z p ) m , with the inner product defined 
componentwise as the dot product a ■ a = YaLi a % a %^ with all operations in the prime field Z p . 

Similarly, if A is the set F n of all n-tuples over F, then its dual space A may also be taken as 
F n , and the inner product may again be defined componentwise as 



{a, a) =^2< 

3=1 



j=i i=i 



Given a complex-valued function / : A — > C defined on A, its Fourier transform is defined as 
the complex-valued function F : A — > C that maps a £ A to 

F(a) = Y J f(a)^' a \ «GA 

where w = e 2m l p is a primitive complex pth root of unity. 

If we view f = {/(a) : a G A} as a column vector indexed by A, and similarly F = {F(a) : a S A} 
as a column vector indexed by A, then the transform can be expressed in matrix form as 

F = F A i, 

where the Fourier transform matrix Fa is defined as {u/ a,a ^ : d £ A, a G A}. Note that J 7 ^ is 
symmetric; i.e., F\ = J^, where T\ denotes the transpose of Fa- 

In a normal factor graph, a Fourier transform may be simply represented as in Figure [71 The 
transform F(a) is obtained by summing over A, which in this case amounts to a matrix multipli- 
cation. Note that as a factor in a factor graph, we do not have to distinguish between Fa and its 
transpose; Fa is simply a function of the two variables corresponding to the two incident edges, 
and matrix can act on either variable. 
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Figure 7: Normal factor graph of a Fourier transform. 
From the orthogonality relation 
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1^1, a = 0; 
0, a / 0, 
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we obtain the matrix equation 



T\T A = \A\I A , 



where F\ = {u ^ a ' a ^ : a E A, a G A} is the conjugate transpose of Fa, and 1a is the identity matrix 

~' — l-AI -1 .^. Thus we obtain the inverse Fourier 



over A. In other words, the inverse of Fa is F A 
transform 



F -1 F 
j- A r 



\A\ ' 



We say that f and F are a Fourier transform pair, written f f>F. 

More generally, we have the following orthogonality relation (which even more generally applies 
to orthogonal subgroups of finite abelian groups): 

Orthogonality relation for subspaces. Let B be any subspace of a finite subspace A, and let 
B L be the orthogonal subspace of its dual space A; i.e., B 1 - is the subset of elements a G A such 
that (a, a) = for all a G B. Then 



aeB 



(a,a) 



\B\, aeB- 
0, a $ B~ 



Proof: Obvious for a G -B -1 -. For a ^ -B^, let z(d) = Y2aeB > and let a' be some element of B 



J2aeB 

0. 



(a,a'+a) 



u^z(a) 



but (a, a') ^ 
□ 



such that (a, a') ^ 0. Since a' + B = B, we have z(o) 
implies u/ a,a ) ^ 1, so z(a) = u/ a ' a ^ z(a) implies z(a) 

Let &b ■ A — > {0, 1} be the indicator function of the subspace B; i.e., ^^(a) = 1 if a G -B, else 
3>B( a ) = 0. From this orthogonality relation, it follows that the Fourier transform of &b is |-B|$ B ±. 
Conversely, by a similar argument, the inverse Fourier transform of is |-B _l | < I ) b/|A|. Thus if B 
and B 1 - are orthogonal subspaces of a finite vector space A and its dual space A, respectively, then 

• the indicator functions $b and are a Fourier transform pair, up to scale; 

• \B\\B ± \ = \A\ = \A\. 

As [12] notes, the top result is a version of the "picket-fence miracle" [I] (see also |13l Theorem 9]). 
We observe that the Poisson summation formula, namely 



15- 



aeB 



aeB J 



follows directly from the Fourier transform relationship for orthogonal code indicator functions, 
where the functions / o F are any Fourier transform pair; see Figure (Note that since both <3?# 
and $ B i are real-valued, the Fourier transform relation holds with the kernel T\ as well as with 
Fa ) We conclude that the Fourier transform pair relationship <3?£ f-> 3> B ± is more fundamental 
than the Poisson summation formula, which is the usual starting point in the development of 
MacWilliams identities. 
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Figure 8: Proof of the Poisson summation formula. 
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Finally, we extend these definitions to a set of indeterminates x = {x(a) : a G ^4} indexed by A, 
rather than a complex- valued function. The transform of this set is then a dual set of indeterminates 
X = {X(a) : a £ A} indexed by A, defined by 

X = J-^x. 

Again, we have the inverse transform relationship 



x = J7 X 



\A\ ' 



and we say that x and X are a transform pair, written x f-> X. Figure [9] shows the corresponding 
normal factor graph, similar to Figure [7J 



X 


A 




A 







X 



A 



Figure 9: Normal factor graph of a set x of indeterminates indexed by A, and the dual set X. 

For example, if A = Z2, then X(0) = x(0) + x(l) and X(l) = x(0) — x(l); similarly, x(0) ■ 
ipT(O) + X(l)), and x(l) = |(X(0) - X(l)). For another example, if A = (Z 2 ) 2 , then 

X(00) = x(00) + x(10) + x(01) + x(ll) 

X(10) = x(00) - x(10) + x(01) - x(ll) 

X(Q1) = x(00) + x(10) -x(01) -x(ll) 

X(ll) = x(00) -x(10) -x(01) +x(ll) 



and vice versa (up to a scale factor of 4). 



3.2 Dual linear codes 

Again, a linear code C over a finite field F is a subspace C C „4 of a symbol sequence space 
.4. = OfceZA wnere eac h symbol alphabet A^ is a finite-dimensional vector space over F. 

As we have seen, each vector space A^ has a dual space A^ of the same dimension such that for 
all £ Ak,ak € A^ there is a well-defined inner product (dfc,^). The dual space to the symbol 
sequence space A = Ilfceiu is then the dual symbol sequence space A = Y\k e x A W1 th the 
inner product defined componentwise by (a, a) = Ylk&x A ^k-,dk) (where we continue to assume 
that is finite). The dual (orthogonal) code C A to C C is then the set of all dual symbol 
sequences a. £ A such that (a, a) = for all a G C. 

From our general orthogonality theorem, we have that the indicator functions <3?c an d $c x are 
a Fourier transform pair, up to scale, and that |C | IC -1 - 1 = |_4|. 

Since the inner product is defined componentwise, i.e., (a, a) = YlkeXi ^ follows that 

the Fourier transform may be taken separately with respect to each variable Ak involved in C. 
For example, consider the indicator function $c fe of a constraint code Ck in a conventional state 
realization, as in Figure [2j The indicator function for the orthogonal code C Sk x x Sfe+i 
may be obtained (up to scale) by transforming with respect to each of the three incident variables 
separately, as shown in the normal factor graph of Figure [101 
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Figure 10: Transform of constraint code indicator function <I>c fe in a conventional state realization. 



3.3 Fourier transform identities 

In this subsection we develop a few quick and easy facts about the algebra of Fourier transforms. 

We have already noted that J 7 a J 7 a = -Taj up to a scale factor of \A\. We will write such a 
relationship as FaF\5 oc I a- (Alternatively, we could normalize each Fourier transform matrix by 
an appropriate scale factor, namely l/yp4| for J- a-) 

We next observe that 

\A\, a + a' = 0; 
0, a + a'^0, 



agA 



by the basic orthogonality relation. In other words, FJFa = |^4|$~A, where &^a is the sign 
inverter indicator function on A; i.e., the indicator function of the sign inversion relation, a' = —a. 

More simply, as a factor in a normal factor graph, the concatenation of J- a with itself is equiv- 
alent up to scale to provided that the alphabet at each end of the chain is A; see the top line 
of Figure [TT1 In this sense, we may write {J- a) 2 oc &~a, where the product indicates concatenation. 
(Note that if the alphabet at each end of the chain were A, then (J- a) 2 would be equivalent up to 
scale to $^^4, the sign inverter indicator function on A.) 

In the same sense, it is easy to see that (Fa) 3 oc F^, and (.Fa) 4 oc Ia- In other words, Fa is a 
fourth root of unity under concatenation, up to scale, and indeed behaves very much like i = 
Similarly, T\ oc (Fa) 3 is a fourth root of unity that is conjugate to Fa, while <3?^a oc (Fa) 2 is a 
square root of unity and is real (equal to its conjugate). 

Figure [TT1 illustrates these relationships. In this figure, edges are labelled simply by the alphabet 
of the associated variable rather than by the variable itself. 
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Figure 11: Fourier transform identities. 



We remark that the relation Ia oc (Fa) 4 oc J-a^^^a is an instance of the Fourier transform 
relationship for orthogonal code indicator functions, since for any vector space A and its dual A, 
the orthogonal code to the sign inverter code = {(a, —a) : a G ^4}, whose indicator function is 
&^ A , is the repetition code C=a = {(a, a) : a G A}, whose indicator function is Ia- 

Finally, note that in the special case where the characteristic of F is p = 2, we have that 
T\ = Fa and <!>^a = I A) *- e -; Fa becomes a square root of unity, and the sign inverter relation 
becomes the equality relation, a' = a. 
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3.4 Normal factor graph duality theorem for linear codes 

We will now prove the normal graph duality theorem for linear codes by dualizing a normal factor 
graph. We continue to ignore scale factors. 

Again, we start with a normal factor graph whose partition function is the indicator function 
<3?c of a linear code CCi, with half-edges representing external variables {^4fc}, edges representing 
internal variables {Sj}, and vertices representing indicator functions {^c^}- 

To obtain a normal factor graph whose partition function is the indicator function $ c ± of 
the orthogonal linear code C 1 - C A, we apply appropriate Fourier transforms J- to each of the 
external variables A/., as in Figure 9. For example, Figure PT21 illustrates this transformation for two 
consecutive sections of a conventional state realization as in Figure [2j 
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Figure 12: Normal factor graph representing the orthogonal code to that of Figure [2 

Next, using the concatenation relation / oc J-&^J- illustrated in Figure [TT1 we replace each 
edge by the concatenation of an appropriate transform matrix J 7 , sign inverter indicator function 
<£^, and transform matrix T. For example, Figure [131 illustrates this replacement for the normal 
factor graph of Figure CGI Here Sk and S' k are two equal state variables, and dual state variables 
on opposite sides of sign inverters have been given opposite signs. 
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Figure 13: Second step in dualizing Figure [2j 

We then observe that every constraint code indicator function is now surrounded by Fourier 
transforms, as in Figure [TOl and therefore we may replace it by the indicator function of the dual 
constraint code, as shown in Figure [TU Note that the sign inverter indicator functions remain. 
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Figure 14: Final step in dualizing Figure 2. 
In summary, we have proved: 
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Normal factor graph duality theorem for linear codes. Given a normal factor graph whose 
partition function is the indicator function <I>c of a linear code C, comprising symbol alphabets 
{Ak} associated with half-edges, state spaces {Sj} associated with ordinary edges, and constraint 
code indicator functions {«$Ci} associated with vertices, the dual normal factor graph is defined 
by replacing each symbol alphabet Ak by its dual symbol alphabet Ak, each state space S& by its 
dual state space Sk, each indicator function <&c- by the dual indicator function <& c ±, and finally 

i 

by placing a sign inverter indicator function in the middle of every ordinary edge. Then the 
partition function of the dual normal factor graph is equal to <& c ± , up to scale. □ 

This theorem is equivalent to our original normal graph duality theorem |5j: given a normal 
graph representing a linear code C, comprising symbol alphabets {Ak} associated with half-edges, 
state spaces {Sj} associated with ordinary edges, and constraint codes {Ci} associated with vertices, 
the dual normal graph is defined by replacing each symbol alphabet Ak by its dual symbol alphabet 
Ak, each state space Sk by its dual state space Sk, each constraint code Ci by its orthogonal code 
C^~, and finally by placing a sign inverter in the middle of every ordinary edge. Then the dual 
normal graph represents the orthogonal code . 

For example, Figure [15] shows the dual to the normal graph of a conventional state realization 
that was shown in Figure [TJ 
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Figure 15: Dual normal graph for a conventional state realization. 

The following two examples illustrate how to dualize binary and nonbinary convolutional codes 
by dualizing the local constraint codes Ck- Example 3 involves a rate- 1/2, four-state binary convo- 
lutional code that is often used in examples. Example 4 involves the rate-2/3 ternary convolutional 
code that was the principal example of Gluesing-Luerssen and Schneider [9| [TO]. 

Example 3 (binary linear convolutional code). Consider the rate-1/2 binary linear time-invariant 
convolutional code C generated by the degree-2 generators (1 + D 2 ,l + D + D 2 ), in standard 
D-transform notation. In other words, C is the set of all output sequences of the single-input, 
two-output linear time-invariant system over F2 whose impulse response is (11, 01, 11, 00, . . .). This 
system has a conventional state realization as in Figure Q] in which each symbol alphabet Ak may 
be taken as (F2) 2 , each state space Sk may also be taken as (F2) 2 , and each constraint code Ck is 
the (6, 3) binary linear block code generated by the three transitions 

00 11 10 
10 01 01 

01 11 00 

which represent the three nontrivial (state, symbol, next state) transitions in the impulse response 
of the system. (Note that only the output symbols appear in C; the input symbols that would 
appear in an input-state-output realization are here regarded as internal variables, and do not 
appear explicitly.) The eight codewords of Ck are the eight possible transitions of the system, 
which are shown as a "trellis section" in Figure [ToT a). with the three generating transitions dashed. 
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Figure 16: Trellis sections of (a) rate-1/2 binary convolutional code; (b) orthogonal code. 

The orthogonal code may easily be seen to be the (6, 3) binary linear block code generated 
by the three transitions 

00 11 01 

01 10 10 
10 11 00 

which represent the three nontrivial (state, symbol, next state) transitions in the impulse response of 
a system with impulse response (11, 10, 11, 00, . . .), or (l + D + D 2 , l + D 2 ) in D-transform notation. 
This is indeed the generator of the orthogonal convolutional code C 1 - under the symbolwise definition 
of the inner product that we are using here. (For the more usual sequencewise definition of the 
inner product, we need to take the time reversal of C^H which in this case happens to be the 
code generated by (1 + D + D 2 , 1 + D 2 ) again.) The eight codewords of are the eight possible 
transitions of the orthogonal system, which are again shown as a "trellis section" in Figure [TOlfb). 
with the three generating transitions dashed. □ 

Example 4 (ternary linear convolutional code; cf. [9J [TO]). Consider the rate-2/3 linear time- 
invariant convolutional code C over F3 with gi(D) = (1 + D 2 ,2 + D,0) and g2^D) = (1,0,2). In 
other words, C is the set of all output sequences of the two-input, three-output linear time-invariant 
system over F3 whose impulse responses are (120, 010, 100, 000, . . .) and (102, 000, . . .). This system 
has a conventional nine-state realization as in Figure [1] in which each symbol alphabet Ak may be 
taken as (F3) 3 , each state space Sk may be taken as (F3) 2 , and each constraint code Ck is the (7,4) 
ternary linear block code generated by the four generators 

00 120 10 
10 010 01 

01 100 00 
00 102 00 

which represent the four nontrivial (s^, cifc, Sk+i) transitions in the two impulse responses of the 
system. The orthogonal code C^r is the (7, 3) ternary linear block code generated by the three 
generators 

00 010 12 

21 202 11 

22 111 00 

The symbolwise inner product of two sequences a, b £ A is J^ fc aubk, and that of a and a shift of b by j time 
units is ^2 k cikbk-j- The product of the corresponding D-transforms a(D) — £\ a kD k and b(D~ 1 ) = bkD~ k is 
X^ tXfc a,kbk-j)D : ' , so a is orthogonal to all shifts of b if and only if a(D)b(D~ 1 ) — 0, or equivalently if and only 
if the sequencewise inner product a(D)b(D) is zero, where b(D) is the D-transform of the time-reversed sequence 
b = {b-k : k 6 Ia}. Thus T> is the orthogonal code to a linear time-invariant code C under the symbolwise inner 
product if and only if the time-reversed code T> is the orthogonal code to C under the sequencewise inner product. 
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which represent the three nontrivial (§k, — %+i) transitions in the impulse response of a con- 
ventional state realization of a single-input, three-output linear system over F3, with sign inverters, 
whose impulse response is (010, 202, 111, 000, . . .), or (2D + D 2 , 1 + D 2 , 2D + D 2 ) in D-transform 
notation. (Note the unconventional basis of the dual state space, and the effects of the sign 
inversions.) This is indeed the generator of the orthogonal convolutional code C 1 - under our sym- 
bolwise definition of the inner product. (For the more usual sequencewise definition of the inner 
product, we need to take the time reversal of C^, which in this case is the code generated by 



3.5 General normal factor graph duality theorem 

Finally, we note that although we have been focussing on normal factor graphs whose factors and 
partition functions are indicator functions of linear codes, the proof of the normal factor graph 
duality theorem in the previous subsection does not depend upon this restriction. Thus we have 
actually proved a much more general theorem: 

General normal factor graph duality theorem. Given an arbitrary normal factor graph with 
partition function Z(a), up to scale, comprising external variables {A^} associated with half-edges, 
internal variables {Sj} associated with ordinary edges, and factors {/j} associated with vertices, 
the dual normal factor graph is defined by replacing each external variable by its dual variable 
Afc, each internal variable Sj by its dual variable Sj, each factor /j by its Fourier transform /j, and 
finally by placing a sign inverter indicator function <I>^ in the middle of every ordinary edge. Then 
the partition function of the dual normal factor graph is the Fourier transform Z(k) of Z(a), up to 
scale. (Note: pp shows that the scale factor is \Sj\-) □ 

Mao and Kschischang [15] have proved a closely related duality theorem. In their development, 
a (multiplicative) factor graph represents a product of factors, rather than a sum of products. 
They show that the dual to a multiplicative factor graph is a convolutional factor graph, which 
represents the convolution of its factors. If the elements of a factor graph are dualized as above 
(but without the sign inverters), then a factor graph and its dual represent a Fourier transform 
pair, up to scale. Using this duality result and the fact that the indicator functions of a linear code 
and its dual are a Fourier transform pair, they derive a factor graph duality theorem for linear 
codes. They then specialize to normal factor graphs, and show how to recover our original normal 
graph duality theorem for linear codes. A similar derivation would yield the general normal factor 
graph duality theorem above. The advantage of our approach is that by focussing on normal factor 
graphs and their partition functions from the beginning, we do not need to introduce convolutional 
factor graphs (which may however prove to have their uses). 

Al-Bashabsheh and Mao [1] have also proved this theorem by their methods, independently and 
at about the same time. Loeliger et al. [131 Appendix III] have proved an analogous theorem for 
continuous alphabets. 




□ 
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4 Mac Williams identities 

Given these duality results, we can obtain various Mac Williams identities, using similar derivations. 

4.1 MacWilliams identities for exact weight generating functions 

We will first consider what MacWilliams and Sloane [2] call exact weight generating functions, 
where every symbol alphabet A k is given its own set of indeterminates x k = {xk(a k ) ■ a>k G A k }- The 
dual set of indeterminates = {X^ia^) : £ A k } is then given by the transform X^. = J-^Xfc. 
The exact weight generating function (EWGF) of C is the multivariate polynomial 

^(x) = 5>(a) = 5>c(a)x(a), 

aeC aeA 

where x(a) = H keXA x k (a k ). 

Similarly, the EWGF of C 1 - is the multivariate polynomial 

5 *(X) = J> c x(a)X(a). 

By the Poisson summation formula, these dual EWGFs are equal, up to scale; see Figure [T71 (Again 
we note that since both <3?c an d are real-valued, the Fourier transform relation holds with the 
kernel J-^ as well as with J-jy.) Explicitly, we have the following MacWilliams identity showing 
how the EWGF of may be computed from that of C: 

^x(X)oc^(J"*X) 
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Figure 17: Duality of exact weight generating functions. 



Now again suppose that we have a normal factor graph whose partition function is <3?c ( U P 
to scale), defined by a set A = {A k } of symbol alphabets, a set S = {Sj} of state spaces, and 
a set {^Ci} °f constraint code indicator functions, each constraint code Cj constraining subsets 
Aj C A, Sj C S of the symbol and state alphabets, respectively. Then we may obtain a normal 
factor graph whose partition function is the global EWGF gj? (x) by connecting each half-edge 
representing A k to a corresponding indeterminate function x k : a k \— > x k (ak), as in Figure El The 
local exact weight generating function <7^(xj,Sj) is then obtained by summing over Aj for each Cj 
to obtain 

5^(xj,Sj)= ^c l (ai,Si)xi(aj). 
The global EWGF is then the resulting global partition function, up to scale: 
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In the case of a conventional state realization, the local EWGF is a matrix indexed by Sk x Sk+i, 

Af (x k ) = {gc k (xk,s k ,s k+ i) : s k G S k ,s k+1 G S k+1 }, 

which we will call the exact weight adjacency matrix (EWAM), and the global EWGF is the matrix 
product of these EWAMs, as was shown for HWGFs and HWAMs in Figures H] and [5j 

In general, if the graph of a code is a tree (i.e., cycle-free), then since a global generating function 
of the code is a sum of products of local factors, it may be computed by a straightforward application 
of the generic sum-product algorithm [5J, in which the "messages" are generating functions of 
subtrees. Even if the graph is not a tree, the global generating function is still the sum of the 
products of the local generating functions over all s G S. Consequently one method of computing 
it is to cut just enough state edges so that the graph becomes a tree (a minimal spanning tree), 
compute the global generating function for this tree using the sum-product algorithm, and then 
sum over the remaining state variables. 

Example 5 (tail-biting trellis). A tail-biting trellis consists of a chain of N trellis sections, with 
the further constraint that the final state sn is equal to the initial state sq. To compute a global 
generating function of a tail-biting trellis, we may thus first form the matrix product of the N 
constituent local WAMs to obtain a global WAM, indexed by (sq, sat); we then sum over all elements 
of this WAM for which so = sn; i.e., we take the trace of the global WAM. □ 

To obtain a MacWilliams identity for local EWGFs, we proceed as follows. Each local EWGF 
gJf,(x4,Si) is the partition function of the local graph fragment shown in Figure [T8fa). comprising 
a local constraint code indicator function attached via an edge labelled by the local symbol 
alphabets Aj to an exact weight generator function Xj. Here we partition the local state spaces Sj 
into two subsets Sf and S^~, such that globally every state space Sj appears once in a plus subset 
and once in a minus subset (corresponding to the sign inversion in the dual normal factor graph). 
Thus the local EWGF will now be written as g^. (xj , s+, s^~). 

Proceeding again along the lines of the derivation of the Poisson summation formula (see Figure 
|5J) , we now replace the function Xj by the concatenation of Xj and an appropriate inverse Fourier 
transform J 7 *; we adjoin appropriate inverse Fourier transforms J 7 * to each Sj G Sf; and we adjoin 
appropriate inverse Fourier transforms J-* and sign inverter indicator functions <3?^ to each Sj G S~, 
as shown in Figure fT8T b) . Because the concatenation of J 7 *, <3?^ and T* is the identity (see Figure 
[TT]) . and because globally every state space Sj appears once in a plus subset and once in a minus 
subset, this will leave the global EWGF unchanged. 

Finally, noticing that each code constraint indicator function &a is now surrounded by (inverse) 
Fourier transforms, we may replace the whole ensemble by the orthogonal code indicator function 
3Vx, as shown in Figure ITSfc). Globally, it is evident that we now have a factor graph whose 

i 

partition function is g^ ± (X), comprising the dual normal factor graph for $^±, with each half-edge 
representing Aj connected to a corresponding indeterminate function Xj. 

Now we have the following MacWilliams identity, showing how the local EWGF of Cj~ may be 
computed from that of Cf. 

5 ^(Xi,S+ Sr)ocg|(Jl i X i! ^+S+ J- s -(-Sr)), 
where J-"*_S^ oc T s - (— S^~) since the variables in Sf are the same as those in some other — S~~. 
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Figure 18: Dualizing local exact weight generating functions. 



In particular, with a conventional state realization with constraint codes Ck, with the convention 
that Sk is involved in with a positive sign and Sk+i is involved in with a negative sign, we 
have the following Mac Williams identity, which shows how the EWAM Afc(X&) of C^ may be 
obtained from the EWAM A fc (x fc ) of C k : 

Af (X fc ) cx (^|J T Af (J-l fc X fe )(Js fc+1 ) T 

(Here the transposes are needed to conform to standard matrix conventions, even though T = J 7 ^.) 

Example 3 (cont.). Continuing with the rate-^ binary convolutional code of Example 3, each 
constraint code Ck has the exact weight adjacency matrix below, where we write xoo, x oi> £io> £11 
for Xfc(OO), Xfe(Ol), Xfc(lO), Xfc(ll), respectively. 





00 


10 


01 


11 


00 


£00 










10 








^01 


^10 


01 


x\\ 


£oo 








11 








£10 


£01 



Given this EWAM for Ck, the EWAM Aj^(Xfc) of the orthogonal constraint code C^r is given by the 
matrix equation below, where we have substituted the dual indeterminates Aoo, Aio, Aoi and An 
for xoo + £io + £oi + £n, £oo - £io + £oi - £n, £oo + £io - £oi - £n and x o - £io - £oi + £n, 
respectively. (We have also inserted the correct scale factor.) 





" 1 


1 1 


1 ~ 




xoo £n 




" 1 1 1 1 " 




Aoo 


An 
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-1 1 - 
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x i £io 
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An 


A 00 
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1 -1 - 


-1 




xn xoo 




1 1-1-1 







Aio 


Aoi 




1 - 


-1 -1 


1 _ 




xio x i _ 




1-1-1 1 _ 







A i 


Aio 


From Fi| 


mrefToTb). 


we see that this matrix is indeed the EWAM of the constraint 


code C^. 





Of course it is no surprise that a dual constraint code C 4 is determined by Cj, or vice versa] what 
the Mac Williams identities give us is a way of calculating one from the other, by taking transforms. 

4.2 Mac Williams identities for other weight generating functions 

Commonly each symbol alphabet Ak is equal to ¥ nk , the set of nfc-tuples over the ground field 
F; i.e., a k = {aki 6 F : 1< f < n^}. In this case a transform over Ak may be expressed by rik 
transforms over F applied to the nk components a,kt of Ak- 
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We may then be interested in the complete weight generating function (CWGF) of C, defined 



as 



at A \kex A e.=i 

where x = {x(a) : a £ F} is a set of indeterminates defined on F. In other words, the CWGF is 
obtained by substituting the product n™=i x ( a ^) f° r x fc(°fc) m the EWGF. 
Similarly, the local complete weight generating function g^,(xi, Sj) is 



9Ct ( x > s i 



3-i G A-i 



,fcel Ai 



where C X4 denotes the subset of indices of symbol alphabets A k that are involved in Cj. 

By a development parallel to that above, we have the following MacWilliams identity, which 
shows how the local CWGF of Cf~ may be computed from that of Cf. 

g c c± (X, S+ Sr) oc ggffiX, ^+ S ^ S - (-Sr)), 

where X = J-jp-x is the dual set of indeterminates to the set x, with J-jr the transform matrix on F. 

In particular, for conventional state realizations, the following MacWilliams identity shows how 
the complete weight adjacency matrix (CWAM) A^(X) of Cj^r may be obtained from the CWAM 
A<?(x)ofC fc : 

A£(X fc ) oc (^) T A^(J|X)(J- Sfc+1 ) T . 

Example 3 (cont.). For the rate-^ binary convolutional code of Example 3, each constraint code 
C k has the CWAM 



a£(x) 



Sk/Sk+l 


00 


10 


01 


11 


00 
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x 


T 2 

x 1 








10 








XQXl 


XQXl 


01 
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r 2 
x Q 








11 








x x 1 





where we have written xq and x\ instead of x(0) and x(l), respectively. The CWAM A^(X) of the 
orthogonal constraint code is given by the matrix equation below, where we have substituted 
the dual indeterminates Xq and X\ for xq + x\ and xq — x%, respectively. 



1 1 

1 -1 

-1 -1 

-1 1 



Xq X 1 













•1 



' 







XqX\ XqXi 



1 1 
1 -1 
1 1 
1 -1 



*o 2 










*1 2 





*l 








X0X1 





XqX\ 





XqXi 





XqX\ 



□ 



We see from Figure [TBTb) that this matrix is indeed the CWAM of C^r . 

Here the point is that even though the CWGF of Cj does not fully determine Cj, it does determine 
the CWGF of C^, and vice versa. 

The Hamming weight generating function (HWGF) g^{x) of a linear code C may be obtained 
by substituting 1 for x(0) and x for each x(a),a 7^ 0, in ^(x), and similarly for the Hamming 
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weight adjacency matrix (HWAM) A¥(x) of a constraint code Ck in a conventional state realization. 
Thus each element of Au(x) becomes a polynomial of degree rik or less in the single indeterminate 
x. The dual indeterminates become X(0) = 1 + (|F| — l)x and X(a) = 1 — x for d / 0, which 
scale to 1 and X = (1 — x)/(l + (|F| — l)x), respectively. Substituting in the above Mac Williams 
identities for CWGFs or CWAMs, we obtain MacWilliams identities for HWGFs or HWAMs. This 
yields the main result of [9j [10] 

Example 3 (cont.). For the rate-1/2 binary convolutional code C of Example 3, each constraint 
code C k has the HWAM 



A 



Hi 



s/s' 
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For the orthogonal code C^~, each constraint code C^r has the HWAM 



A H (X) 



s/s' 


00 


10 


01 


11 


00 


1 





X 1 





10 


X 1 
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01 





X 





X 


11 





X 





X 



The reader may verify that A H (x) and A H (X) satisfy a MacWilliams identity with X = jt^- 
[Hint: it may be easier to start with CWAMs.] Note that here A H (x) happens to be the transpose 
ofA H (x). □ 

Example 4 (cont.). For a worked-out example of the HWAM Af:(X) of the orthogonal code C^r 
to the constraint code Ck of Example 4, see [10] . □ 

Again, the point is that even though the HWGF of does not fully determine Cj, it does 
determine the HWGF of C^~, and vice versa. 

Ericson and Zinoviev |3j have generalized this method of obtaining further weight generating 
functions from complete weight generating functions, as follows. A partition of a finite abelian 
group G into disjoint subsets {Gj} and of its character group G into disjoint subsets {Gj} is called 
a Fourier-invariant pair if for all i,j the transform of the indicator function &(Gj) of Gj, namely 

$ti) = Y,*(G j ){g,g)= 52 &g), 



depends only on the subset Gi that contains g, and similarly for the inverse transform, where 
(<?><?) = 9(9)- For example, the Hamming partition pair defined by Go = {0},Gi = G \ {0} and 
Go = {0}, Gi = G\{0} is Fourier- invariant. For any Fourier-invariant partition pair, a MacWilliams 
identity may be obtained for the corresponding generalized weight generating function; see [3] or 
0]. Zinoviev and Ericson [T7] show that this concept is equivalent to that of an association scheme. 



6 The MacWilliams identity of [9] [10] is stated in terms of the HWAM for a minimal realization of a linear 
time-invariant convolutional code C in controller canonical form, and the HWAM of some minimal encoder for the 
orthogonal code C x . Our results apply to the CWAM or HWAM of any state realization, and the CWAM or HWAM 
of its dual realization, because in our development, by constraint code duality, the basis of the dual state space 
representation is fixed as soon as the basis of the primal state space is fixed. 
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5 Mac Williams identities for terminated convolutional codes 



A principal use of Mac Williams identities is to compute the distance distribution of a linear code 
C from that of its dual code , where typically C is high-rate and C is low-rate (i.e., \C- L \ < \C\). 

For a convolutional code C, the most commonly studied distance distribution is its free (Ham- 
ming) distance spectrum, namely, the distribution of (Hamming) weights of codewords in C that 
start and end in the zero state without passing through an intermediate zero stateQ Shearer and 
McEliece [16] showed by example that the free distance spectrum of C does not in general determine 
that of C ± , and therefore that there could be no Mac Williams identity for such distributions. 

Recently, Bocharova, Hug, Johannesson and Kudryashov [2] have proved a Mac Williams identity 
for truncations of a convolutional code C and its orthogonal code C^~. By letting the truncation 
length become large, they obtain an approximation to the free distance spectrum of C. 

In this section, we derive similar results for weight distributions of codes obtained by various 
kinds of termination procedures, of which we regard tail-biting as the most elegant. We argue that 
these alternative distributions are as useful for estimating code performance as the free distance 
spectrum. These results effectively answer the original question posed by Shearer and McEliece 
|16j . which we would state as follows: is there a duality relationship that allows us to estimate the 
performance parameters of from those of C? 

5.1 Terminated convolutional codes 

A general method for approximating the free distance spectrum of a linear convolutional code C 
is to derive a series of block codes Cat of length N from C by some sort of termination procedure, 
and then to study the distance distributions of Cn as N — > oo. As we will see, for any of the 
termination methods below, the distance distribution of Cm, normalized by N, approaches the 
free distance spectrum of C for df ree < d < 2df ree , where df ree denotes the free distance of C (the 
least weight of any nonzero code sequence). However, we will argue that tail-biting is the nicest, 
particularly if we are also interested in the distance distribution of the orthogonal convolutional 
code C ± . 

The left side of Figure [T9l shows normal graphs of five block codes Cn obtained from a convo- 
lutional code C by five kinds of termination procedures: 

(a) the subcode C[n,jv); 

(b) the projection Ci[o,jV)5 

(c) the truncated code C^ro^)! 

(d) the reverse-truncated code C>[o,jv)' and 

(e) the tail-biting code Ciirtyv)- 

In each case the central part of the graph consists of N consecutive trellis sections of C, and the 
block code symbols are the corresponding convolutional code symbols (ao, • • • , «n-i)- 

To obtain the subcode C^at), the starting and ending state variables are constrained to be zero: 
so = sn = 0. For the projection Ci[o j\r), the starting and ending states may be any arbitrary pair 

7 We will assume in this section that the unique state sequence associated with the all-zero code sequence is the 
all-zero state sequence; this assumption can always be satisfied by choosing a minimal encoder for C. 
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so G Sq,sn G Sat. For the truncated code C^at), we constrain sq = 0, but let sat be arbitrary; 
for the reverse-truncated code, the reverse constraints are imposed. Finally, for the tail-biting code 
C||[o,A0> we i m P ose the constraint sn = sq. 
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Figure 19: Terminated convolutional codes and their duals (over a field of characteristic 2). 

The right side of Figure [19] shows the orthogonal codes to all of these codes, obtained simply 
by use of the normal graph duality theorem, along with the observation that the orthogonal code 
to a trivial code {0} is a universe code, e.g., So, and vice versa. (For simplicity, we have assumed 
that the characteristic of F is 2, so that we do not need to show sign inverters.) 

We observe that the orthogonal code to a subcode C[o,at) of C is the corresponding projection 
(C -1 - ) | [o,A^) °f C^-, and vice versa, as is well known; the orthogonal code to a truncated code C < [ 0i at) 
of C is the corresponding reverse-truncated code {C^)^^) °f C , and vice versa [2j; and the 
orthogonal code to a tail-biting code C||[ n) of C is the corresponding tail-biting code (C^mo m of 

Since C [0)N) and (C^^at), or C <[0 ,n) and (C- L ) >[0)A r), or C||[ ,JV) and (C-^iko.ao are orthogonal 
block codes, there is a Mac Williams identity between their respective weight generating functions. 

We now give examples that will illustrate these various kinds of terminated codes, and the 
following general properties: 

• The rate of a subcode C[q,aO ^ s ^ ess than that of C, the rate of a projection is higher than that 
of C, and the rate of a truncated or a tail-biting code is the same as that of C. 

• The minimum distance of a subcode C^at) ^ s (at least) the same as the minimum free distance 
(if ree of C. As is well known, a tail-biting code has the same property, provided that N is large 
enough. However, the other kinds of terminated codes in general have low-weight codewords. 
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Example 3 (cont.). We now consider various methods of terminating the rate-^ binary convolu- 
tional code C of Example 3 with a block length of N = 4. 

The subcode C[o,4) is the (8, 2) binary linear block code generated by the two generators 



11 01 11 00 
00 11 01 11 

The minimum distance of this block code is the same as the minimum free distance c?f rcc = 5 of C, 
although its rate is lower. 

The orthogonal code to the subcode C[o,4) is the projection (C^)^) of the orthogonal convolu- 
tional code which is the (8,6) binary linear block code generated by the six generators 

11 00 00 00 

10 11 00 00 

11 10 11 00 
00 11 10 11 
00 00 11 10 
00 00 00 11 

The minimum distance of this block code is 2, less than the minimum free distance ofC ± , although 
its rate is higher. 

The truncated code C<[o,4) is the (8,4) binary linear block code generated by 

11 01 11 00 
00 11 01 11 
00 00 11 01 
00 00 00 11 

The minimum distance of this block code is 2, but its rate is the same as that of C. Its orthogonal 
code (C" L ) I> [o ) 4) is the (8,4) binary linear block code generated by 

11 00 00 00 

10 11 00 00 

11 10 11 00 
00 11 10 11 

which has the same parameters. 

The tail-biting terminated code C||[o,4) is the (8,4) binary linear block code generated by 

11 01 11 00 

00 11 01 11 
11 00 11 01 

01 11 00 11 

whereas the orthogonal tail-biting terminated code (C^n^AO is the (8,4) binary linear block code 
generated by the four generators 

11 10 11 00 

00 11 10 11 

11 00 11 10 

10 11 00 11 
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Both of these codes have a minimum distance of only 2 (e.g., for paths such as 01 00 01 00 from 
state 10 to state 10 in Cii[o,4))- However, for N > 10, it turns out that the minimum distance of 
both tail-biting terminated codes is 5, the same as the minimum free distance of C or C ± . □ 

5.2 Distance distributions of convolutional codes and terminated codes 

In this subsection, we consider how the free distance spectrum of a linear time-invariant convolu- 
tional code C may be derived from the weight distribution of any of these terminated codes of length 
N as N — > oo. Then, in the next subsection, we show how the weight generating functions of any of 
these terminated codes may be determined from the weight adjacency matrix of the convolutional 
code. Again, the most elegant relationships are obtained for tail-biting terminated codes. 

We continue to assume that the unique state sequence associated with the infinite all-zero code 
sequence is the all-zero state sequence. Consequently, the lowest-weight words of a terminated code 
as N — > oo must be those that pass through the zero state almost all of the time. These code 
sequences are as follows, for the various termination methods we have considered: 

• If we terminate to the subcode C^tv), then code sequences start and end in the zero state, and 
the lowest-weight sequences correspond to the lowest-weight sequences in the free distance 
spectrum. If the minimum free distance is <if r ee> then for df rec < d < 2df rec there will be 
approximately N x sequences in the terminated code of weight d, where is the number 
of code sequences of weight d in the free distance spectrum of C. Thus, for df TCC < d < 2d{ ree , 
the weight distribution per unit time of C is the limit of the weight distribution of C[o,iV) 
normalized by (divided by) N as N — > oo. For d > 2df Tce , there will be overcounting — e.g., 
two sequences of weight df ree may be counted as one of weight 2df ree — but we will argue below 
that such overcounting should not affect estimates of code performance. 

• If we terminate to the projection C|[o,jv), then code sequences can start and end in any state, 
and there will be low-weight sequences starting with a low-weight state transition s — > 0, 
remaining in state for nearly N time units, and then ending with a low- weight transition 
— > s', where s and s' are not both 0. Thus the minimum distance of C|[o,jv) wm be less than 
df ree for all TV. However, the number of such low-weight sequences remains constant, so after 
normalization we will eventually see the same normalized weight distribution as for C[o,jv)- 

• If we terminate to the truncated code C<[o,at), then by the same argument we will eventually 
see the correct normalized weight distribution. In this case, for a code sequence that starts 
in the zero state, remains there for nearly N time units, and then ends with a low-weight 
transition — > s, the total weight is only that of the low-weight transition — > s. However, 
again the number of such low- weight sequences remains constant, so after normalization we 
will eventually see the correct normalized weight distribution. 

• If we terminate to the tail-biting code C||[o,jv)) then by the same argument we will eventually 
see the correct normalized weight distribution. Note however that in this case the total weight 
of a code sequence starting with a low-weight transition s — > 0, remaining in the zero state 
for nearly N time units, and then ending with a low-weight transition — > s, must be at least 
dfree, since the ending sequence (corresponding to the state transition — > s) followed by the 
starting sequence (corresponding to s — > 0) must be a cyclic shift of a code sequence of C. 
Thus the minimum distance of C||[o,jv) must equal df ree for large enough N. 
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We conclude that as N — > oo the normalized weight distribution of any of these terminated 
codes approaches the free distance spectrum of C for df rcc < d < 2df ree . However, only the tail-biting 
code has the same rate as C and the same minimum distance df ree (for large enough N). 

We now argue that the normalized weight distribution of any of these terminated codes Cn 
must yield the same estimate of code performance over N time units as the free distance spectrum 
of C, if these estimates are accurate. The probability of error event P(£) of C per unit time may be 
estimated using the free distance spectrum. The probability of any error in N time units is then 
estimated as NP(£). If this is a good estimate (implying N < \/P{£)), then the probability of 
two or more error events in N time units must be negligible. But the probability of any error in 
decoding C over N time units is essentially the same as the probability of block decoding error in 
decoding Cn, which may be estimated by the weight distribution of N, which counts codewords 
that include two or more error events. If the probability of two or more error events in N time 
units is negligible, then an estimate based on the weight distribution of Cn must approximately 
agree with an estimate based on the free distance spectrum of C. 

5.3 Free distance spectra for convolutional codes from terminated codes 

We now show how weight generating functions for terminations of a linear time-invariant convo- 
lutional code C may be derived from the weight adjacency matrix of the constraint code C^ that 
specifies C. This will allow us to state Mac Williams identities for terminated convolutional codes, 
and to estimate code performance. 

We compute the Hamming weight distributions of these terminated codes as follows. Let 
A-[o,7V)( x ) be the Hamming weight adjacency matrix of C over the interval [0, TV), whose elements 
are indexed by Sq x Sn- As we have seen in Example 2, if C is time-invariant and A(x) is the 
HWAM of each constraint code then Ai 0}N )(x) is simply equal to A N (x). 

From their definitions, we see that the HWGFs of various terminated codes of C can be read as 
follows from the HWAM A N (x): 

(a) The HWGF of the subcode C [0 ,iV) is the (0,0) element of A N (x). 

(b) The HWGF of the projection Ci[o,jv) is the sum of all elements of A N (x). 

(c) The HWGF of the truncated code C<j[o,jv) is the sum of all elements in the first row of A N (x). 

(d) The HWGF of the reverse-truncated code C>[o,at) is the sum of all elements in the first column 



its trace Tr (A (x)) (see Example 5). 

Example 3 (cont.). For the rate- 1/2 binary convolutional code C of Example 3, the HWAM of a 
section consisting of TV = 2 time units of our example code C is thus 



(e) 




(x); i.e., 



A 2 



(x) = 



1 x 2 x 3 X 3 

3 2 2 

2 4 
XXX X 

X 3 X X 2 X 2 



.3 -I 
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This shows that there is exactly one path from each state in Sk to each state in Sk+2, and that the 
minimum Hamming weight of any of these paths (other than the zero path) is 1. 
For a section consisting of N = 4 time units of this code, the HWAM is 



A 4 (x) 



1 + 2x 5 + x 6 
x 3 + 2x A + x 5 
■ 2 + x 3 + x 4 + a 
x 3 + 2x 4 + x 5 



x 2 + X 3 + X 4 + X 7 

x 2 + x 3 + x 5 + x 6 



+ x 4 + 2x E 



X 2 + X 3 + X 5 + X 6 



x 3 + 2x 4 + x E 
2x 3 + x 4 + x 6 
2 + x 3 + x 5 + 
2x 3 + x 4 + x 6 



x 3 + 2x 4 + x 5 
2x 3 + x 4 + x 6 
2 + x 3 + x 5 + a 
2x 3 + x 4 + x 6 



This shows that there are four paths from each state in Sk to each state in Sk+4, and that the 
minimum nonzero Hamming weight of any of these paths is 2. 

The Hamming weight generating function of the tail-biting termination C||[o,4) of length 4 is 
the trace of A 4 (x), namely 1 + 2x 2 + 4x 3 + x 4 + 4x 5 + 4x 6 . Since A 4 (x) happens to be the 
transpose of A 4 (x), the orthogonal tail-biting terminated code (C^)||[o,4) has the same Hamming 
weight generating function. It is easy to check that the Hamming weight generating function of 
this code is indeed invariant under the MacWilliams transform. □ 

Using tail-biting terminated codes, and normalizing the weight distribution by dividing by N, 
we have that the generating function of the normalized Hamming weight distribution of C is 

9c (x) = ton iTr (A n (x)). 

N— >oo iv 

Moreover, there is a MacWilliams identity between gc(x) and <7c±(x). The performance of C may 
be estimated from gc(x), and that of from g c ±(x). (Similar observations are made in [2], using 
truncated codes.) 

It appears that the behavior of gc(x) might be analyzed by using an extension of Perron- 
Frobenius theory to generating function matrices, as in [8]; however, we have not attempted such 
an analysis. 

Example 1 (cont.). For a section consisting of N = 16 time units of the rate- 1/2 binary convolu- 
tional code C of Example 1, the HWAM A 16 (x) (modulo x 8 ) is 



1 + 14a; 6 



25a; 



-44a:' 



x 3 + 2a; 4 + 4a; 5 + 8a; 6 + 16a; 7 
x 2 + x 3 + 2x 4 + Ax 5 + Sx G + 29x r 
x 3 + 2x 4 + 4a; 5 + 8a; 6 + 16a; 7 

Notice that 



: 3 + 2x A + 4a: 5 + 8x 6 + 29x 7 

x 5 + 3a; 6 + 8a; 7 
r 4 + 2a- 5 + 5a; 6 + 12a' 7 

a- 5 + 3a' 6 + 8a' 7 



- 2a; 4 + 4a: 5 + 8a: 6 + 16a; 7 x 3 + 2x 4 + 4a: 5 + 8a: 6 



-4x' 



-4x' 



' + 3x 6 + 8x 7 



' + 3x 6 + 8x 7 



■4x' 



-4a:' 



Tr (A 16 (x)) = 1 + 16x 5 + 32x 6 + 64x 7 H , 

so that normalizing the distribution by dividing the higher-order coefficients by iV = 16 already 
gives the precise free distance spectrum of C for d < 8, namely x 5 + 2x 6 + 4x 7 + • • - . Thus the 
convergence to the limiting generating function gc{x) is rapid and exact. This property of tail-biting 
codes is not shared by other kinds of terminations. □ 
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Example 6 (c/. |16tl2]). The two codes proposed by Shearer and McEliece [16] for their counterex- 
ample provide an excellent final example. The first code is a rate-1/3 binary linear time-invariant 
convolutional code C\ generated by the degree-1 generators (1, 1 + D,D), i.e., C\ is generated by 
a minimal encoder with impulse response is (110,011,000, . . .), whose trellis section is shown in 
Figure [20^a). The HWAM of this encoder is 



HOD rm rm QQQ 





wi — 



(a) (b) 
Figure 20: Trellis sections of (a) rate-1/3 2-state binary convolutional code C\\ (b) similar code C^- 

The second code is a rate-1/3 binary linear time-invariant convolutional code C2 generated by 
the degree-1 generators (D,D, 1 + D), i.e., C2 is generated by a minimal encoder whose impulse 
response is (001, 111,000, . . .), whose trellis section is shown in Figure 1207b). The HWAM of this 
encoder is 

yV ( 1 1 *c 

Since the weights of the — > and 1 — > 1 transitions are the same for C\ and C2, and since 
the sums of the weights of the — > 1 and 1 — > transitions are the same, it is evident that the 
weight distributions of the subcodes (Ci)[o,jv) an d (^2)[q,n) are the same for all N, and that the 
free distance spectra of C\ and C2 are also the same. For the same reason, the weight distributions 
of the tail-biting codes (Ci)ii[o jv) and (C2)||[o,iV) are the same for all N. 

However, the weight distributions of the projections (Ci)i[o,jV) an d (C2)\\o,N) are n °t the same 
even for N = 1. It follows that the weight distributions of the subcodes (Ci~)[o,iv) and (C^Oro jv) of 
their orthogonal rate-2/3 codes and are not the same, and therefore that their free distance 
spectra are not the same; this was the point of Shearer and McEliece [16j . 

On the other hand, since the weight distributions of the tail-biting codes (Ci)||[o,jv) and (C2)||[o,Ar) 
are the same for all N, it follows that the weight distributions of the tail-biting codes (Ci~)||[o,jv) an< ^ 
(C2 )||[o,jv) are th e same for all N . 

Since the performance of and C^ - may be estimated from these weight distributions as 
N — > 00, it follows that the performance of Ci and is effectively the same, despite the difference 
in their free distance spectralfl □ 

In summary, similarly to j2j. we have shown that there is a Mac Williams identity between 
the generating functions of the weight distributions per unit time of a linear convolutional code 
C and its orthogonal code (as calculated from their tail-biting terminations), and that these 
distributions are as useful as their free distance spectra for estimating code performance. These 
results effectively answer the question posed by Shearer and McEliece [IB] . 



8 Another way of reaching the same (or a stronger) conclusion is to observe that Ci and C2 are equivalent under a 
simple finite-memory permutation. Therefore Ci and C2 must be equivalent under the same permutation, and thus 
must have precisely the same performance on a memoryless channel with maximum likelihood decoding. 
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